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PCT Art. 19 Amendment 

CLAIMS 

1. (Amended) A video coding apparatus comprising: 
a coder section for generating coded information 

obtained by arbitrary shape coding an input video 
picture as a bit stream; 

an important information constructing section for 
extracting, from the coded information obtained by said 
coder section, important information representing a 
rule of coding a set of predetermined bit streams, and 
constructing the important information; 

a sync signal generator section for generating a 
sync signal; and 

a bit stream reconstructing section for adding the 
sync signal output from said sync signal generator 
means, an HEC code representing whether or not the 
important information is duplexed and the important 
information constructed by said important information 
constructing section to the bit stream coded by said 
coder section, to reconstruct the bit stream. 

2. (Amended) The video coding apparatus according 
to claim 1, wherein said important information 
constructing section comprises: 

* a first important information constructing section 
for constructing, from the coded information, 
conventional picture relating information as important 
information in a conventional coding scheme of coding a 
frame in units of rectangular regions; 
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a second important information constructing 
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section for constructing, from the coded information, 
arbitrary shape picture relating important information 
as important information in an arbitrary shape coding 
scheme of coding a picture in the frame in units of 
5 arbitrary shape picture regions; 

an arbitrary shape coding determination section 
for determining based on the coded information whether 
the coded picture is an arbitrary shape picture; 

a switch section for outputting the arbitrary 
10 shape picture relating important information when said 

arbitrary shape coding determination section determines 
that the coded picture is an arbitrary shape picture; 
and 

a multiplexer section for multiplexing the 
15 conventional picture relating information and an output 

from said switch section. 

3. The video coding apparatus according to 
claim 2, wherein said arbitrary shape coding 
determination section determines whether the coded 

20 picture is a rectangular picture or an arbitrary shape 

picture to output a determination signal and controls 
said switch section in accordance with the 
determination signal . 

4 . The video coding apparatus according to 

25 claim 1, wherein said bit stream reconstructing section 

duplexes the important information in a predetermined 
format and insert the duplexed important information 
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into a header. 

5. The video coding apparatus according to 
claim 1, wherein said bit stream reconstructing section 
comprises a macroblock boundary determination section 
for determining whether or not the coded bit stream is 
positioned on a macroblock boundary, a counter for 
counting bits of the coded bit stream, a sync signal 
inserting determination section for outputting an 
insertion permission signal when the macroblock 
boundary determination section determines the 
macroblock boundary for the bit stream and a count 
value of the bits of the bit stream exceeds a given 
value, a header information inserting section for 
forming expansion header information from the important 
information and the sync signal and adding the 
expansion information to the coded bit stream in 
response to the insertion permission signal from said 
sync signal inserting determination section. 

6. The video coding apparatus according to 
claim 5, wherein the expansion header information is 
provided in a video packet starting at the sync signal. 

7. (Amended) A video decoding apparatus 
comprising : 

a bit stream receiving section for receiving a bit 
stream including coded video information that is 
obtained by arbitrary shape coding a video picture, 
sync information and header information including 
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important information representing a rule of coding a 
set of predetermined 



bit streams; 

a demultiplexer section for demultiplexing the 
coded bit stream into a picture bit stream 
corresponding to the video information; 

a decoder section for decoding the picture bit 
stream; 

a sync signal detector section for detecting a 
sync signal from the bit stream, and informing said 
decoder section of the sync signal; 

an error check section for checking presence of an 
error on the basis of decoded information of said 
decoder section; and 

an important information constructing section for 
constructing important information from the header 
information output from said decoder section, and 
informing said decoder section of the important 
information when said error check section determines 
absence of an error. 

8. (Amended) The video decoding apparatus 
according to claim 7, wherein said important 
information constructing section comprises: 

a first important information constructing section 
for constructing conventional picture relating 
important information from the header information; 

an arbitrary shape coding determination means for 
determining based on the header information whether a 



picture decoded from the header information is an 
arbitrary shape picture; 

a second important information constructing means 
for constructing arbitrary shape picture relating 
important information from the header information; 

a first switch section for inputting the header 
information to said second important information 
constructing section when said arbitrary shape coding 
determination section determines that the decoded 
picture is an arbitrary shape picture; and 

a second switch section for outputting the 
arbitrary shape picture relating important information 
when said arbitrary shape coding determination section 
determines that the decoded picture is an arbitrary 
shape picture. 

9. (Amended) The video decoding apparatus 
according to claim 7, wherein said important 
information constructing section outputs VOP (video 
object plane) header information when a VOP header is 
included in the bit stream obtained during decoding of 
said decoder section, and outputs the important 
information of the VOP when no VOP header is included 
in the bit stream obtained during decoding of said 
decoder section. 

10. (Amended) The video decoding apparatus accord- 
ing to claim 9, wherein said important information 
constructing section decodes the conventional picture 
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relating important information when the VOP header is 
found from the bit stream in said decoder section. 

11. The video decoding apparatus according to 
claim 7, wherein said decoder section executes a 

5 decoding operation from a position of a next sync 

signal detected by said sync signal detector section 
after performing a processing corresponding to an error 
when said error check section detects the error. 

12. (Amended) A video coding apparatus comprising: 
10 a coder section for generating coded information 

obtained by arbitrary shape coding an input video 
picture as a bit stream; 

an important information constructing section for 
extracting, from the coded information obtained by said 
15 coder section, important information representing a 

rule of coding a set of predetermined bit streams, and 
constructing the important information; 

a bit stream divider section for dividing the bit 
stream coded by said coder section into a plurality of 
20 divided bit strings; 

a packet header generator section for generating a 
packet header from the important information 
constructed by said important information constructing 
section; and 

25 a packet constructing section for constructing a 

packet using the divided bit strings and the packet 
header. 
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13. (Amended) The video coding apparatus according 
to claim 12, wherein said important information 
constructing section comprises: 

an arbitrary shape picture relating important 
5 information constructing section for constructing 

arbitrary shape picture relating important information 
from the coded information; 

an expansion header insertion determining section 
for determining based on the coded information whether 
10 an expansion header holding the arbitrary shape picture 

relating important information is to be inserted in the 
packet header; and 

a switch section for inputting the header 
information to the arbitrary shape picture relating 
15 important information when said expansion header 

insertion determining section determines that the 
expansion header is to be inserted, 

14. The video coding apparatus according to 
claim 12, wherein said packet header generator section 

20 outputs the packet header containing the important 

information therein to said packet constructing section, 
and said bit stream divider section divides the bit 
stream output from said coder section in a packet size 
and outputs it. 

25 15. (Amended) A video decoding apparatus comprising: 

a bit stream receiving section for receiving a bit 
stream including coded video information obtained by 
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arbitrary shape coding a video picture, sync 
information, and packet header information including 
important information representing a rule of coding a 
set of predetermined bit streams when the video picture 
5 is coded; 

a demultiplexer section for demultiplexing the bit 
stream into a picture bit string of the video picture 
information and packet header information; 

a decoder section for decoding the picture bit 
10 string; 

an error check section for checking presence of an 
error on the basis of decoded information of said 
decoder section; and 

an important information constructing section for 
15 extracting and constructing the important information 

from the packet header information, and informing said 
decoder section of the constructed important 
information, when said error check section determines 
presence of an error and the important information is 
20 required in decoding. 

16. (Amended) The video coding apparatus according 
to claim 15, wherein said important information 
constructing section comprises: 

■ a first important information constructing section 
25 for constructing arbitrary shape picture relating im- 

portant information from the packet header information; 
an expansion header insertion determining section 
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for determining based on the packet header information 
whether an expansion header is inserted in the packet 
header; and 

a switch section for inserting the packet header 
5 information to the arbitrary shape picture relating 

important information when said expansion header 
insertion determining section determines that the 
expansion header is inserted. 

17. (Amended) A video transmission system 
10 comprising: 

a server computer including a video coding 
apparatus for arbitrary shape coding a video to 
generate a coded bit stream; 

a transmitter for transmitting the coded bit 
15 stream of said server computer; 

a receiver for receiving the coded bit stream from 
said server computer; and 

a client computer including a video decoding 
apparatus for decoding the coded bit stream from said 
20 receiver, and wherein 

said video coding apparatus comprises: 
a coder section for generating coded information 
obtained by coding an input video picture as a bit 
stream; 

25 an important information constructing section for 

extracting, from the coded information obtained by said 
coder section, important information representing a 
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rule of coding a set of predetermined bit streams, 
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and constructing the important information; 

a sync signal generator section for generating a 
sync signal; and 

a bit stream reconstructing section for adding the 
sync signal output from said sync signal generator 
means, an HEC code representing whether or not the 
important information is duplexed and the important 
information constructed by said important information 
constructing section to the bit stream coded by said 
coder section, to reconstruct the bit stream, and 

said video decoding apparatus comprises: 

a demultiplexer section for demultiplexing the 
received coded bit stream into a picture bit stream 
corresponding to the video information; 

a decoder section for decoding the picture bit 
stream; 

a sync signal detector section for detecting a 
sync signal from the bit stream, and informing said 
decoder section of the sync signal; 

an error check section for checking presence of an 
error on the basis of decoded information of said 
decoder section; and 

an important information constructing section for 
constructing important information from the header 
information output from said decoder section, and 
informing said decoder section of the important 
information when said error check section determines 
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absence of an error. 
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18. (Amended) A video transmission system 
comprising : 

a server computer including a video coding 
apparatus for arbitrary shape coding a video to 
5 generate a coded bit stream; 

a transmitter for transmitting the coded bit 
stream of said server computer; 

a receiver for receiving the coded bit stream from 
said server computer; and 
10 a client computer including a video decoding 

apparatus for decoding the coded bit stream from said 
receiver, and wherein 

said video coding apparatus comprises: 
a coder section for generating coded information 
15 obtained by arbitrary shape coding an input video 
picture as a bit stream; 

an important information constructing section for 
extracting, from the coded information obtained by said 
coder section, important information representing a 
20 rule of coding a set of predetermined bit streams, and 
constructing the important information; 

a bit stream divider section for dividing the bit 
stream coded by said coder section into a plurality of 
divided bit strings; 
25 a packet header generator section for generating a 

packet header from the important information constructed 
by said important information constructing section; and 
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a packet constructing section for constructing a 
packet using the divided bit strings and the packet 
header, and 

said video decoding apparatus comprises: 
5 a demultiplexer section for demultiplexing the 

received bit stream into a picture bit string of the 
video picture information and packet header 
information; 

a decoder section for decoding the picture bit 
10 string; 

an error check section for checking presence of an 
error on the basis of decoded information of said 
decoder section; and 

an important information constructing section for 
15 extracting and constructing the important information 
from the packet header information, and informing said 
decoder section of the constructed important 
information, when said error check section determines 
presence of an error and the important information is 
20 required in decoding. 

19. (Amended) A recording medium storing: 
a header including information that is used in 
common to coded data; and 

a VOP including: a first video packet that 
25 comprises a VOP header and macrobook data; and at least 
one second video packet that comprises a VP header and 
macrobook data is a shape information header storing 
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arbitrary information of a picture to be recorded, 

said VP header including a sync signal, and an HEC 
code representing whether or not the important 
information is duplexed, the important information 
5 indicating a rule according to which coded information 
obtained by arbitrary shape coding is coded. 

20. The recording medium according to claim 19, 
wherein the important information of the VP header 
includes information about a width of a picture, 

10 information about a height of the picture, information 
about x-and y coordinates at which the picture is 
pasted, a flag representing whether coding is done 
after the shape information is scaled-down, and 
information about a coding type of the shape 

15 information. 

21. (Amended) A video decoding apparatus 
comprising : 

demultiplexer means for demultiplexing an input 
bit stream into a picture stream; 
20 decoder means for decoding the picture bit stream; 

sync signal detector means for detecting a sync 
signal from the picture bit stream, and informing said 
decoder means of the sync signal; and 

important information constructing means for 
25 constructing important information from header 
information output from said decoder means, and 
informing said decoder means of the important 
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nf ormation . 

22. (Amended) A video coding apparatus accordin 
o claim 21, further comprising: 

means for constructing conventional picture 
elating important information from the header 
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information; 

arbitrary shape coding determination means for 
determining based on the header information whether a 
decoded picture is an arbitrary shape picture; 
5 arbitrary shape picture relating important 

information constructing means for constructing 
arbitrary shape picture relating important information 
from the header informations- 
switch means for inputting the header information 
10 to said arbitrary shape picture relating information 
constructing means when said arbitrary shape coding 
determination means determines that the decoded picture 
is an arbitrary shape picture; and 

switch means for outputting the arbitrary shape 
15 picture relating important information when said 

arbitrary shape coding determination means determines 
that the decoded picture is an arbitrary shape picture. 
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23. (New) A video coding apparatus according to 
claim 1, characterized in that said important 
information includes information representing an image 
size and an image position. 

24. (New) A video coding apparatus according to 
claim 7, characterized in that said important 
information includes information representing an image 
size and an image position. 

25. (New) A video coding apparatus according to 
claim 12, characterized in that said important 
information includes information representing an image 
size and an image position. 

26. (New) A video coding apparatus according to 
claim 15, characterized in that said important 
information includes information representing an image 
size and an image position. 

27. (New) A video coding apparatus according to 
claim 17, characterized in that said important 
information includes information representing an image 
size and an image position . 

28. (New) A video coding apparatus according to 
claim 18, characterized in that said important 
information includes information representing an image 
size and an image position . 

29. (New) A video coding apparatus according to 
claim 19, characterized in that said VP header includes 
an HEC code representing that the important information 



> 



• » 



81/2 



is duplexed, and important information relating to 
duplexed arbitrary shaped codes. 
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Title of Invention: Moving Picture Encoding Apparatus and Moving Picture 
Decoding Apparatus 

As the result of examination of the present application, the following 
reasons for rejection have been found and notified herein on the basis of the 
provision of Section 63 of the Patent Law. Any opinion about the rejection 
[Form 25-2 attached to the Regulations under the Patent Law], and/or any 
amendment [Form 5 attached to the Regulations under the Patent Law] must be 
filed by August 24, 2004. (The above date is extensible by one month for each 
request. No notification of allowance of extension of time will be issued.) 

[Reason] 

The inventions recited in claims 1, 2, 4, 7-9, 11,17, 21-24, 27, 28, 30-32, 
and 34-47 are unpatentable under Section 29 (2) of the Patent Law, as being 
such that the invention could easily have been made by a person with ordinary 
skill in the art to which the invention pertains, on the basis of the invention 
described in the following publication distributed prior to this application 



[Remarks] 



Claims 1,2,4, 23, 30 and 35-38 claim a moving picture encoding 
apparatus (method) comprising an encoding section (step), an important 
information constructing section (step), a sync signal generating section (step), 
and a bit string constructing section (step). Claims 7, 8, 9, 1 1, 24, 31 and 39- 
42 claim a moving picture decoding apparatus (method) comprising a bit string 
receiving section (step), a separating section (step), a decoding section (step), 
a sync signal detecting section (step) and an important information constructing 
section (step). Claims 17, 27 and 32 claim a moving picture transmission 
system comprising a server computer including a moving picture encoding 
apparatus, a transmitter, a receiver and a client computer including a moving 
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picture decoding apparatus. Claims 43-45 claim a moving picture transmission 
method comprising a moving picture encoding step in a server computer, a 
transmitting step, a receiving step and a moving picture decoding step in a 
client computer. Claims 21 , 22, 28, 34, 46 and 47 claim a moving picture 
decoding apparatus (method) comprising separating means (step), decoding 
means (step), sync signal detecting means (step) and important information 
constructing means (step). Addition of important information and a sync signal 
to an image signal bit string in an encoding and decoding apparatus (method) is 
the same as the addition of important information and a sync signal to an image 
signal bit string described in "World of MPEG-4", edited by S. Miki, published by 
Kogyo Chosakai in Japan on September 30, 1998 (hereinafter referred to as 
"Cited Invention 1", a Korean translation thereof "World of MPEG-4" translated 
by Kou Sonze and Kim Zonoku, published by Daieisha is attached). The 
technique of transmitting important data in duplicate and the use of a header 
diffusion code as information relating thereto is the same as the use of a header 
diffusion code of the error restoration tools in MPEG-4 described in Tallui, "Error 
resilient video coding in the ISO MPEG-4 standard", IEEE Comm. Magazine, 
June 1998, pp 1 12-1 19 (hereinafter referred to as "Cited Invention 2"). It is 
merely well-known and commonly-used art in this field of art to transmit and 
receive coded information and exchange information on the position and size of 
an image by moving picture encoding between server and client computers. 
The size, position and mode flag, etc. recited in claims 30-34, 38, 42, 45 and 47 
are nothing more than those used in the MPEG-4 standard, as described in the 
detailed description of the invention. 

Therefore, claims 1, 2, 4, 7-9, 11,17, 21-24, 27, 28, 30-32, and 34-47 
are obtained by simply combining Cited Invention 1, Cited Invention 2, the 
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MPEG-4 standard and the well-known and commonly-used art as described 
above. It is not particularly difficult to combine them. It is not considered that 
a new advanced effect can be obtained by the combination. Consequently, the 
present invention could have been easily achieved on the basis of Cited 
Invention 1, Cited Invention 2, the MPEG-4 standard and the well-known and 
commonly-used art (Section 29 (2) of the Patent Law). 

[Attachment] 

Attachment 1 . A copy of Tallui, "Error resilient video coding in the ISO MPEG-4 
standard" IEEE Comm. Magazine, June 1998, pp 112-119 

June 24, 2004 
Korean Patent Office 
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THE FOLLOWING IS THE ENGLISH TRANSLATION OF THE 
ANNEXES TO THE INTERNATIONAL PRELIMINARY 
EXAMINATION REPORT: AMENDED SHEETS (Pages 3-8, 11-42, 
53-54, 68-81/9, Figures 1, 2, 4, 20 and 26). 



in the video decoding apparatus greatly degrades. 

More specifically, in transmitting a 
compression-coded picture using a communication network, 
the receiving side must execute decoding processing of 
reconstructing significant information from a 
transmitted "0"/ n l" bit stream. 

For this purpose, the above-described header 
information is very important as information 
representing the rule of coding a set of predetermined 
bit steams. Examples of the header information are 
information representing the prediction type of frame 
being decoded (whether intraframe coding or interframe 
coding) , information (time reference) representing the 
display timing of the frame, and step size information 
used in performing quantization. 

If these pieces of header information are lost, 
information transmitted subsequently cannot be normally 
decoded. 

For example, assume that an error is mixed in a 
bit stream owing to any cause, and the bit pattern 
changes to represent intraframe coding though the 
prediction type of frame is supposed to represent 
interframe coding. In this case, even if subsequent 
actual information is normally transmitted, the 
decoding side determines the signal as the result of 
intraframe coding, and cannot normally decode the 
signal at last. 
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Consequently, the quality of a reconstructed video 
picture in the video coding apparatus greatly degrades. 

Mixture of errors frequently occurs in a system, 
such as a radio videophone, portable information 
5 terminal, or radio digital television receiver, that 

transmits and/or stores a video picture via a radio 
transmission line . 

The mainstream of conventional picture 
transmission is a system using a cable communication 
10 network. Even in the use of a radio communication 

network, picture transmission assumes satellite 
communication whose error rate is very low. The 
structure of a coded stream to be transmitted does not 
sufficiently consider the error resilience, and 
15 important information such as header information is not 

satisfactorily protected against the transmission line 
error . 

In a PHS (Portable Handyphone System) expected to 
become one of the mainstreams of future mobile 

20 communication, the error rate is about several hundred 

thousand to million times that of satellite 
communication. Therefore, errors cannot be fully 
corrected only by conventional error correction done 
for a coded bit stream. 

25 In the Internet expected to become one of the 

mainstreams of future communication as well as the PHS, 
time at which an error is mixed and the type of mixed 
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error are not statistically clarified, and no proper 
error correction may be done. 

However, the conventional construction has 
problems that the information capable of doubling using 
HEC includes no information required for coding 
arbitrary shape picture, and thus when VOP header 
information is lost by transmission error while the 
arbitrary shape picture is decoded in units of object, 
the arbitrary shape picture cannot be correctly decoded. 

For this rea'son, in transmitting a code stream 
coded using arbitrary shape picture coding, the error 
resilience of transmission data weakens. 

It can be attained by the present invention to 
make even in an arbitrary shape picture coding to have 
an error resilience similar to that in a conventional 
coding method for coding a rectangular picture. 

Disclosure of Invention 

The first invention provides a video coding 
apparatus comprising a coder unit for coding an input 
video picture to obtain a bit stream, an important 
information constructing unit for extracting, from 
coded information obtained by the coder unit, important 
information as information representing a rule of 
coding a set of predetermined bit streams, thereby 
constructing the important information, a sync signal 
generator unit for generating a sync signal, and a bit 
stream reconstructing unit for adding the sync signal 
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output from the sync signal generator unit and the 
important information reconstructed by the important 
information constructing unit to the bit stream coded 
by the coder unit, thereby reconstructing the bit 
5 stream. 

The second invention provides a picture coding 
apparatus wherein the important information 
constructing unit in the first invention comprises a 
conventional picture relating important information 

10 constructing unit for constructing, from the coded 

information, conventional picture relating information 
as important information in a conventional coding 
scheme of coding a frame in units of rectangular 
regions, an arbitrary shape picture relating important 

15 information constructing unit for constructing, from 

the coded information, arbitrary shape picture relating 
important information as important information in an 
arbitrary shape coding scheme of coding a picture in 
the frame in units of arbitrary shape picture regions, 

20 an arbitrary shape coding determination unit for 

determining based on the coded information whether the 
coded picture is an arbitrary shape picture, a switch 
unit for outputting the arbitrary shape picture 
relating important information when the arbitrary shape 

25 coding determination unit determines that the coded 

picture is an arbitrary shape picture, and a 
multiplexer unit for multiplexing the conventional 
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picture relating information and an output from the 
switch unit. 

The third invention provides a video decoding 
apparatus for decoding coded data including a bit 
stream including sync information that is obtained by 
coding a video picture, and includes, as header 
information, important information serving as 
information representing a rule of coding a set of 
predetermined bit streams, comprising a demultiplexer 
unit for demultiplexing an input bit stream into a 
picture bit stream, a decoder unit for decoding the 
picture bit stream, a sync signal detector unit for 
detecting a sync signal from the picture bit stream, 
and informing the decoder unit of the sync signal, an 
error check unit for checking presence of an error on 
the basis of decoded information of the decoder unit, 
and an important information constructing unit for 
constructing important information from header 
information output from the decoder unit, and informing 
the decoder unit of the important information when the 
error check unit determines absence of an error. 

The fourth invention provides a video decoding 
apparatus wherein the important information 
constructing unit in the third invention comprises a 
conventional picture relating important information 
constructing unit for constructing conventional picture 
relating important information from the header 
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information, an arbitrary shape coding determination 
unit for determining based on the header information 
whether a decoded picture is an arbitrary shape picture, 
an arbitrary shape picture relating important infor- 
5 mation constructing unit for constructing arbitrary 

shape picture relating important information from the 
header information, a first switch unit for inputting 
the header information to the arbitrary shape picture 
relating information reconstructing unit when the 

10 arbitrary shape coding determination unit determines 

that the decoded picture is an arbitrary shape picture, 
and a second switch unit for outputting the arbitrary 
shape picture relating important information when the 
arbitrary shape coding determination unit determines 

15 that the decoded picture is an arbitrary shape picture. 

The fifth invention provides a video decoding 
apparatus comprising a coder unit for coding an input 
video picture to obtain a bit stream, an important 
information constructing unit for extracting, from 

20 coded information obtained by the coder unit, important 

information as information representing a rule of 
coding a set of predetermined bit streams, thereby 
constructing the important information, a bit stream 
divider unit for dividing the bit stream coded by the 

25 coder unit, a packet header generator unit for 

generating a packet header from the important 
information constructed by the important information 
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According to the present invention, in video 
coding, a header can be set in coded data. At the same 
time, an expansion header can be set in the header to 
include important information in the arbitrary shape 
5 picture coding scheme, in addition to important 

information in the conventional picture coding scheme 
that is stored in the header. Even. if some headers are 
destroyed, a picture can be decoded at a portion having 
normal headers. Further, a sync signal inserted in 

10 picture data can prevent out-of -synchronization of the 

video packet VP. The video packet VP is a packet 
starting from a sync signal RM. Even if an error 
exists before RM to cause out-of -synchronization, this 
sync signal RM can establish resynchronization . 

15 Accordingly, the present invention can provide a 

video coding technigue which attains high noise 
resilience in transmission and can give error 
resilience equivalent to that of conventional 
rectangular picture coding to even arbitrary shape 

20 picture coding. 

Brief Description of Drawings 
FIG. 1 is a block diagram showing the basic 
arrangement of a coder section according to the first 
embodiment of the present invention; 

25 FIG. 2 is a block diagram showing in detail the 

basic arrangements of an important information 
constructing section and bit stream reconstructing 



section in the coder section according to the first 
embodiment of the present invention; 

FIG. 3 is a flow chart showing the basic flow of 
the bit stream reconstructing section according to the 
first embodiment of the present invention; 

FIG . 4 is a view showing the expansion header 
format of VP according to the first embodiment of the 
present invention; 

FIG. 5 is a block diagram showing the basic 
arrangement of a decoder section according to the first 
embodiment of the present invention; 

FIG. 6 is a block diagram showing in detail the 
basic arrangement of an important information 
constructing section in the decoder section according 
to the first embodiment of the present invention; 

FIG. 7 is a table for explaining a coded word 
structure used in the present invention; 

FIG. 8 is a view showing an example for explaining 
the effect of variable-length coding; 

FIG. 9 is a block diagram showing an arrangement 
of performing variable-length coding for important 
information; 

FIG. 10 is a block diagram showing the basic 
arrangement of a coder section according to the second 
embodiment of the present invention; 

FIG. 11 is a block diagram showing the detailed 
arrangement of an important information constructing 
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section in the coder section according to the second 
embodiment of the present invention; 

FIG. 12 is a view showing an example of an 
expansion packet header according to the second 
5 embodiment of the present invention; 

FIG. 13 is a view showing another example of the 
expansion packet header (with a marker) according to 
the second embodiment of the present invention; 

FIG. 14 is a block diagram showing the basic 
10 arrangement of a decoder section according to the 

second embodiment of the present invention; 

FIG. 15 is a block diagram showing the detailed 
arrangement of an important information construction 
section in the decoder section according to the second 
15 embodiment of the present invention; 

FIG. 16 is a view for explaining coding of an 
arbitrary shape picture; 

FIG. 17 is a view for explaining decoding of an 
arbitrary shape picture; 
20 FIG. 18 is a view showing the VOP structure of 

MPEG-4; 

FIG. 19 is a view showing the VP structure of 
MPEG-4 ; 

FIG. 20 is a view showing the VP header format of 
25 MPEG-4 ; 

FIG. 21 shows the problem of conventional VP; 
FIG. 22 shows the effect of conventional VP; 
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FIG. 23 shows the effect of VP using HEC; 

FIG. 24 is a view showing information necessary 
for synthesizing and playing back pictures in decoding 
an arbitrary shape picture; 

FIG. 25 is a view showing an example of a radio 
video transmission system adopting a coding/decoding 
apparatus according to the third embodiment of the 
present invention; 

FIG. 26 is a block diagram showing a coding 
apparatus according to the fourth embodiment that 
corresponds to the coding apparatus according to the 
first embodiment; 

FIG. 27 is a block diagram showing a decoding 
apparatus according to the fourth embodiment that 
corresponds to the decoding apparatus according to the 
first embodiment; 

FIG. 28 is a block diagram showing a decoding 
apparatus according to the fifth embodiment that 
corresponds to the coding apparatus according to the 
second embodiment ; 

FIG. 29 is a block diagram showing a decoding 
apparatus according to the fifth embodiment that 
corresponds to the decoding apparatus according to the 
second embodiment ; and 

FIG. 30 is a flowchart of a decoding apparatus 
according to the third embodiment. 
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Best Mode for Carrying Out of the Invention 
Embodiments of the present invention will be 
described below with reference to the several views of 
the accompanying drawing. 
5 The embodiments of the present invention will be 

described below with reference to the several views of 
the accompanying drawing . 

FIG. 1 shows the basic arrangement of a video 
coding apparatus according to the first embodiment of 

10 the present invention. In FIG. 1, the output of a 

coder section 101 is connected to an important 
information constructing section 102 and bit stream 
reconstructing section 104. The output of the 
important information constructing section 102 is 

15 connected to the bit stream reconstructing section 104 

together with the output of a sync signal generator 
section 103. The output of the bit stream 
reconstructing section 104 is connected to a 
multiplexer section 105. The output of the multiplexer 

20 section 105 is a transmission line 106. 

The coder section 101 codes an input video signal 
131 to output it to the bit stream reconstructing 
section 104, and outputs coded information 133 obtained 
by coding to the important information constructing 

25 section 102. The important information constructing 

section 102 receives the video signal 131 obtained by 
coding in the coder section 101, and selects and 
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outputs only important information 134 necessary for 
decoding . 

The sync signal generator section 103 generates a 
sync signal 135 at an arbitrary interval. The bit 
5 stream reconstructing section 104 inserts the sync 

signal 135 from the sync signal generator section 103 
in a bit stream 132. If necessary, the bit stream 
reconstructing section 104 inserts the important 
information 134 output from the important information 

10 constructing section 102 after the sync signal 135 in 

accordance with a predetermined format, and outputs the 
resultant bit stream. 

The multiplexer section 105 multiplexes a bit 
stream 136 reconstructed by the bit stream 

15 reconstructing section 104 with another data (e.g., 

speech data, or bit stream prepared by coding another 
object) to output a multiplexed bit stream 137 to the 
transmission line/storage medium 106. 

In this arrangement, the input video signal 131 is 

20 coded by the coder section 101. The bit stream 132 

output from the coder section 101 upon coding is input 
to the bit stream reconstructing section 104. The 
coded information 133 obtained by coding in the coder 
section 101 is input to the important information 

25 constructing section 102, which selects and outputs 

only the important information 134 necessary for 
decoding . 
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The bit stream reconstructing section 104 inserts 
in the bit stream 132 the sync signal 135 output from 
the sync signal generator section 103 at an arbitrary 
interval. If necessary, the bit stream reconstructing 
5 section 104 inserts the important information 134 

output from the important information constructing 
section 102 after the sync signal 135 in accordance 
with a predetermined format. 

The bit stream 136 reconstructed by the bit stream 

10 reconstructing section 104 is input to the multiplexer 

section 105 where the bit stream 136 is multiplexed 
with another data (e.g., speech data, or bit stream 
prepared by coding another object) to output the 
multiplexed bit stream 137 to the transmission 

15 line/storage medium 106. 

According to the first embodiment, the sync signal 
135 output from the sync signal generator section 103 
at an arbitrary interval is inserted in a bit stream 
obtained by coding a video picture. If necessary, the 

20 important information 134 output from the important 

information constructing section 102 is inserted after 
the sync signal 135 by the bit stream reconstructing 
section 104 in accordance with a predetermined format. 

The important information constructing section 102 

25 generates, as the important information 134, infor- 

mation necessary for arbitrary shape picture 
coding/decoding in MPEG-4, for example in arbitrary 
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shape picture coding, information about the width VW of 
the picture size, information about the height VH, 
information about the x-coordinate VHMSR of the picture 
position for indicating the display position of a 
decoded picture, information about the y-coordinate 
VVMSR, the VOP shape coding type " vop_shape_coding_type 
(VSCT) " representing the coding mode of shape 
information, and the flag change_conv_ratio_disable 
(CCRD) representing whether coding is done after the 
size of shape information is converted. The pieces of 
important information are duplicated and inserted in a 
VP header by the bit stream reconstructing section 104 
in accordance with a predetermined format, and then 
arbitrary shape picture coding can also attain error 
resilience equivalent to that of rectangular picture 
coding. Even if some VOP headers or VPs are destructed, 
a video picture can be decoded. 

In MPEG-4, a video object plane "Video Object 
Plane" corresponds to the frame (FIG. 18). The Video 
Object Plane (to be referred to as VOP hereinafter) can 
be divided into a plurality of packets, and each packet 
is called a video packet "Video Packet" (FIG. 19) . 

The video packet "Video Packet" (to be referred to 
as VP hereinafter) is a packet starting from a sync 
signal (Resync Marker; to be referred to as RM 
hereinafter) . Even if an error exists before RM to 
cause out-of -synchronization, this sync signal enabled 
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re synchronization . 

Even if, therefore, information is destroyed/lost 
by an error, subsequent video packets VP could be 
normally decoded so long as the video packets VP are 
5 not a start packet. This is because the start VOP 

header of the video object plane VOP has been decoded 
to provide all the pieces of information necessary for 
decoding (FIG. 20) . 

As described above, VOP header information 

10 includes the coding type (intraframe coding, interframe 

coding, and the like) of video object plane VOP, the 
time reference, and the step size. If this information 
is lost, all the video packets VP could not be decoded 
(FIGS. 21A and 21B) . 

15 In MPEG-4, the header extension code HEC is 

defined in the header of the video packet VP, and thus 
thereafter re-describing important information in the 
VOP header enabled based on the HEC value. 

This format is shown in FIGS. 21A and 21B. As 

20 shown in FIG. 21A, the video object plane VOP was 

constructed with a format in which a pattern of a VOP 
header and subsequent data is set at the start, and a 
pattern of a video packet VP header and subsequent data 
is repeated several times. 

25 The header extension code HEC is defined in the VP 

header, and important information in the VOP header is 
described again with the HEC value. Unless the video 
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object plane VOP is destroyed, normal pairs of VP 
headers and data could be decoded using information of 
the VOP header and data even if one or two pairs of the 
headers of video packets VP and subsequent data are 
5 destroyed. 

In the example of FIGS. 22A and 22B, the header of 
the video object plane VOP and its paired data are not 
destroyed, but only the header of the first video 
packet VP and its paired data are destroyed. In this 

10 case, the video object plane VOP and its data are not 

destroyed, as shown in FIG. 22B. Thus, the first 
region of a picture can be normally decoded. In the 
second region, an error occurs to decode a degraded 
picture. The third and subsequent regions of the 

15 picture are normally decoded. The picture could be 

decoded as one which is partially destroyed but almost 
completely reconstructed . 

In a scheme using no header extension code HEC, if 
the VOP header is destroyed, no picture can be decoded 

20 regardless of the remaining normal VPs, as shown in 

FIG. 22A, and no picture can be obtained under the 
influence of the error, as shown in FIG. 22B. Even in 
this case, the picture could be reconstructed by the 
scheme using the header extension code HEC. That is, 

25 in the scheme using the header extension code HEC, when 

the header extension code HEC is true, important 
information is duplicated after HEC; and when HEC is 
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false, important information is not duplicated. In 
using a transmission line whose error rate is high, HEC 
is set to be true, and important information is 
duplicated and added after the HEC. 
5 Even if an error exists at the start of VOP to 

fail in decoding, as shown in FIG. 23A, information 
protected by HEC can be used to decode video packets VP 
in the second and subsequent regions though the picture 
of the start region cannot be normally decoded. The 

10 picture can be decoded as one which is partially 

destroyed but almost completely reconstructed can be 
decoded, as shown in FIG. 23B. 

However, this could only be realized in units of 
rectangular picture regions. More specifically, VOP 

15 header information is duplicated in the VP header using 

HEC. Even when the VOP header is lost, subsequent data 
can be normally decoded using the VOP header so long as 
the VOP header is duplicated in the VP header using HEC. 
However, information which can be duplicated using HEC 

20 does not include any information necessary for 

arbitrary shape picture coding. Although a conven- 
tional rectangular picture can be decoded without any 
problem, a scheme such as MPEG-4 capable of coding an 
arbitrary shape picture in units of objects could not 

25 decode any picture. 

Arbitrary shape picture coding in units of objects 
in MEPG-4 uses a larger number of pieces of header 
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information than in rectangular picture coding. For 
this reason, it was problems that header information 
cannot be duplicated. 

From another viewpoint, as the Internet, intranet, 
5 and the like are becoming popular, these networks are 

often used for communication. Also, Internet 
videophones and the like are being used. In this case, 
a video picture is transmitted in real time. However, 
a video picture transmitted in real time via the 

10 Internet, intranet, or the like poses many problems in 

generally used TCP and UDP protocols. A serious 
problem is that the header does not have any time 
information . 

To prevent this, an RTP (Real-time Transfer 

15 Protcol) recently receives a great deal of attention as 

a protocol used to transmit video picture/speech data. 
That is, a protocol such as TCP does not have any time 
information to each packet, so the receiving side 
cannot obtain the time when received data is 

20 reconstructed. When data is transmitted in units of 

packets, the receiving side cannot satisfactorily 
reconstruct video picture data or speech/sound data. 

However, RTP adds time information to each packet 
to allow the receiving side to reconstruct video 

25 picture data and speech/sound data based on the time 

information. In this manner, RTP is suitable for 
real-time data transmission. 
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This protocol can define an expansion header for 
each application. 

As described in the prior art, MPEG-4 duplicates 
VOP header information in a VP header using HEC. Even 
5 if the VOP header is lost, subsequent data could be 

normally decoded using the VOP header as far as the VOP 
header is duplicated in the VP header using HEC. 

However, information which can be duplicated using 
HEC does not include any information necessary for 
10 arbitrary shape picture coding. A conventional 

rectangular picture can be decoded without any problem. 
To the contrary, arbitrary shape picture coding uses a 
larger number of pieces of header information than in 
rectangular picture coding. Hence, it was serious 
15 problems that header information cannot be duplicated. 

For example, since the picture size changes in 
units of VOPs in arbitrary shape picture coding, a 
width vop_width (to be referred to as VW hereinafter) 
and a height vop_height (to be referred to as VH 
20 hereinafter) of the picture size are described in 

the VOP header. Also, an x-coordinate 
vop_horizontal_mc_spatial_ref (to be referred to 
as VHMSR hereinafter) and a y-coordinate 

vop_vertical_mc_spatial_ref (to be referred to as VVMSR 
25 hereinafter) of the picture position for indicating the 

display position of a decided picture are described. 
The relationship between these values is shown in 
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FIG. 24. 

When a video picture is to be decoded using only 
information of the video packet VP without these pieces 
of information, a picture cannot be normally decoded in 
5 arbitrary shape picture coding. In other words, a 

picture could not be normally decoded in arbitrary 
shape picture coding without information about the 
width VW of the picture size, information about the 
height VH, information about the x-coordinate VHMSR of 

10 the picture position for indicating the display 

position of a decoded picture, and information about 
the y-coordinate VVMSR. 

Normally coding a picture also requires a VOP 
shape coding type " vop_shape_coding_type (to be 

15 referred to as VSCT hereinafter) representing the 

coding mode of shape information, and a flag 
change_conv_ratio_disable (to be referred to as CCRD 
hereinafter) representing whether coding is done after 
the size of shape information is converted. 

20 These pieces of information are not protected in 

duplication of the VOP header using HEC in MPEG-4 . 

In this fashion, the system can give error 
resilience equivalent to that of conventional 
rectangular picture coding to even arbitrary shape 

25 picture coding. The important information constructing 

section 102 and bit stream reconstructing section 104 
as important features of the present invention in the 




above arrangement will be described in detail with 
reference to FIG. 2. 

The important information constructing section 102 
will be explained in detail. 

As shown in FIG. 2, the important information 
constructing section 102 is constituted by a 
conventional picture relating important information 
constructing section 206, arbitrary shape picture 
relating important information constructing section 207, 
arbitrary shape coding determination section 208, and 
multiplexer section 210. 

Of these sections, the conventional picture 
relating important information constructing section 206 
receives the coded information 133 from the coder 
section 101, selects information (e.g., coding mode and 
time reference) determined to be important based on the 
coded information 133 in conventional coding, and 
outputs the information as conventional picture 
relating important information 238 to the multiplexer 
section 210. The arbitrary shape picture relating 
important information constructing section 207 selects 
important information (e.g., picture size, position, 
coding mode, and reduction mode) relating to arbitrary 
shape picture coding, and outputs the information 
as arbitrary shape picture relating important 
information 239. 

The arbitrary shape coding determination section 
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208 determines whether a coded picture is a 
conventional rectangular picture or arbitrary shape 
picture, and outputs the determination result as a 
determination signal 240. 
5 A switch section 209 executes switch control of 

determining based on the determination signal 240 from 
the arbitrary shape coding determination section 208 
whether the arbitrary shape picture relating important 
information 239 from the conventional picture relating 

10 important information constructing section 207 is 

output to the multiplexer section 210. The multiplexer 
section 210 multiplexes the conventional picture 
relating important information 238 from the 
conventional picture relating important information 

15 constructing section 206, and the arbitrary shape 

picture relating important information 239 output 
from the arbitrary shape coding determination section 
208 in arbitrary shape picture coding, and outputs 
the multiplexed information as the important 

20 information 134. 

In this arrangement, the coded information 133 
from the coder section 101 is input to the conventional 
picture relating important information constructing 
section 207 as a construction component of the 

25 important information constructing section 102. The 

conventional picture relating important information 
constructing section 207 selects information (e.g., 



coding mode and time reference) determined to be 
important in conventional coding, and outputs the 
selected information as the conventional picture 
relating important information 238 to the multiplexer 
section 210. As a result, the conventional picture 
relating important information 238 is a set of pieces 
of information such as the coding mode and time 
reference determined to be important in general coding. 

Then, the arbitrary shape picture relating 
important information constructing section 207 selects 
important information (e.g., picture size, position, 
coding mode, and reduction mode) relating to arbitrary- 
shape picture coding, and outputs the selected 
information as arbitrary shape picture relating 
important information 238 to the multiplexer 
section 210. 

The arbitrary shape coding determination section 
208 determines whether a coded picture is a 
conventional rectangular picture or arbitrary shape 
picture, and outputs the determination result as the 
determination signal 240. This determination signal 
240 controls the switch section 209 which performs 
switch control of determining whether the arbitrary 
shape picture relating important information 239 from 
the conventional picture relating important information 
constructing section 207 is output. 

The multiplexer section 210 multiplexes the 
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conventional picture relating important information 238 
and arbitrary shape picture relating important 
information 239 in arbitrary shape picture coding, and 
outputs the multiplexed information as the important 
information 134 . 

As a result, the conventional picture relating 
important information 238 and arbitrary shape picture 
relating important information 239 in arbitrary shape 
picture coding can be output as the multiplexed 
important information 134 from the multiplexer section 
210. In conventional picture coding, only the 
conventional picture relating important information 238 
can be output as the important information 134. 

The bit stream reconstructing section 104 will be 
described in detail. As shown in the upper half of 
FIG. 2, the bit stream reconstructing section 104 is 
comprised of an MB boundary determination section 201, 
counter 202, sync signal inserting determination 
section 203, header information inserting section 205, 
and adder section 206. 

Of these sections, the MB boundary determination 
section 201 determines for the bit stream 132 input 
after being coded by the coder section 101 whether data 
of the bit stream corresponds to the boundary of a 
macroblock MB. The number-of -coded-bits counter 
section 202 counts the number of coded bits of the bit 
stream 132 input after being coded by the preceding 
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coder section 101. 

When the MB boundary determination section 201 
determines that the bit stream 132 corresponds to an MB 
boundary, and the count value of the number-of- 
5 coded-bits counter section 202 for the bit stream 132 

exceeds a given value, the sync signal inserting 
determination section 203 outputs an insertion 
permission signal 234. 

The header information inserting section 205 

10 prepares header information from the input important 

information 134 and sync signal 135. When the sync 
signal inserting determination section 203 determines 
that insertion is permitted, the header information 
inserting section 205 outputs header information 237 

15 prepared for the coded bit stream 132 to the adder 

section 206. 

The adder section 206 adds the bit stream 132 
input after being coded by the coder section 101 to an 
output from the header information inserting section 

20 205, and outputs the sum as the reconstructed bit 

stream 136 of the bit stream reconstructing section 104. 

When the bit stream reconstructing section 104 
having this arrangement receives the bit stream 132 
coded by the coder section 101, the section 104 inputs 

25 the bit stream 132 to the MB boundary determination 

section 201 and number-of -coded-bits counter section 
202 as construction components of the bit stream 
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reconstructing section 104. The MB boundary 
determination section 201 determines whether the input 
bit stream 132 corresponds to an MB boundary. 

The number-of -coded-bits counter section 202 
5 counts the number of coded bits of the bit stream 132. 

When the MB boundary determination section 201 
determines that the bit stream 132 corresponds to an MB 
boundary, and the number of coded bits counted by the 
counter 202 exceeds a given value, the sync signal 

10 inserting determination section 203 generates the 

insertion permission signal 234 to output it to the 
header information inserting section 205. 

The header information inserting section 205 
prepares header information from the input important 

15 information 134 and sync signal 135, and outputs the 

prepared header information 237 to the adder section 
206 in order to add the header information 237 to the 
coded bit stream 132. The adder section 206 inserts 
the header information 237 in the coded bit stream 132 

20 to output the reconstructed bit stream 136. This bit 

stream 136 is output from the bit stream reconstructing 
section 104. 

As a result, the insertion permission signal 234 
is generated if the number of coded bits exceeds a 
25 predetermined value when the bit stream 132 of picture 

data input after being coded by the coder section is 
examined to find that a bit at the boundary position of 
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the macroblock MB is input. Header information 
prepared by the header information inserting section 
205 based on the input important information 134 and 
sync signal 135 can be added to the bit stream 132. 
5 In the important information constructing section 

102, the conventional picture relating important 
information constructing section 206 as one 
construction component selects information (e.g., 
coding mode and time reference) determined to be 

10 important in conventional coding based on the coded 

information 133 from the coder section 101, and outputs 
the selected information as the conventional picture 
relating important information 238. The arbitrary 
shape picture relating important information 

15 constructing section 207 as one construction component 

of the important information constructing section 102 
selects important information (e.g., picture size, 
position, coding mode, and reduction mode) relating to 
arbitrary shape picture coding, and outputs the 

20 selected information as the arbitrary shape picture 

relating important information 239. Only the 
conventional picture relating important information 238 
is obtained for a conventional picture, whereas the 
conventional picture relating important information 238 

25 and arbitrary shape- picture relating important 

information 239 are multiplexed in coding an arbitrary 
shape picture. Therefore, header information inserted 
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in a bit stream can include conventional picture 
relating important information and arbitrary shape 
picture relating important information. The VP header 
can include information necessary to play back the 
5 picture of MPEG-4 coded data. 

FIG. 3 is a flow chart showing preparation of 
header information . 

As the first step (step S502), the bit stream 
reconstructing section 104 determines whether a bit 
10 stream input from the coder section 101 corresponds to. 

the boundary position of MB (MacroBlock) . 

If Y in step S502, whether a sync signal RM is to 
be inserted is determined as the second step (step 
S503) . This determination can be done based on an 
15 arbitrary algorithm of the user. 

For example, various methods can be adopted: an 
algorithm of inserting the sync signal RM when the 
number of bits after a preceding sync signal exceeds a 
predetermined value, or an algorithm of determining 
20 . whether RM is inserted along a picture configuration 

when the number of MBs after a preceding sync signal 
exceeds a predetermined value. 

The video packet VP starts from the sync signal RM. 
Even if an error exists before the sync signal RM to 
25 cause out-of -synchronization, this sync signal RM can 

establish re synchronization. 

If Y in step S503, RM is inserted, and a VP header 
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subsequent to RM is inserted (step S504 in FIG- 3) . 
Then, the flow advances to step S505. 

In the third step (step S505) , whether important 
information of the VOP header is duplicated as an 
5 expansion header is determined. 

If Y in step S505, HEC is true, and important 
information in rectangular picture coding is selected 
and output from the VOP header (step S506 in FIG. 3) . 
The flow shifts to step S507 . 

10 In the last fourth step (step S507) , whether the 

picture is an arbitrary shape picture is determined. 
If Y in step S507, important information in arbitrary 
shape picture coding within the VOP header is selected 
and output (step S508 in FIG. 3) . 

15 The VP header is generated through the four steps, 

and inserted in a bit stream. 

FIG. 4 shows a structure of a VP header in an 
arbitrary shape picture. An expansion header Ex-Header 
is added to the conventional VP header shown in FIG. 20. 

20 This expansion header Ex-Header additionally includes 

important information in arbitrary shape picture coding, 
i.e., the width (VW) and height (VH) of a picture, an 
x-coordinate (VHMSR) and y-coordinate (VVMSR) at which 
the picture is pasted, a flag (CCRD) representing 

25 whether shape information is reduced and coded, and 

information (VSCT) about the coding type (intraframe 
coding, interframe coding, or the like) of picture 
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information . 

Note that important information in arbitrary shape 
picture coding is not limited to the above information. 
Depending on an application purpose, another 
5 information can be added, or information can be reduced. 

However, the transmitting and receiving sides must have 
consensus on the header format. 

The video coding apparatus comprises the function 
of extracting important information in arbitrary shape 

10 picture coding, the function of determining whether an 

arbitrary shape picture is used, and the function of 
detecting the boundary of a macroblock. The VP header 
includes an expansion header which includes a sync 
signal in addition to important information in 

15 conventional picture coding, and important information 

for arbitrary shape picture coding in arbitrary shape 
picture coding. Even if some headers are destroyed, a 
picture can be decoded at a portion having normal 
headers. Further, the sync signal prevents 

20 out-of -synchronization of the video packet VP. That is, 

even if the video packet VP starting from the sync 
signal RM has an error before the sync signal RM to 
cause out-of-synchronization, the sync signal RM can 
establish resynchronization . 

25 Accordingly, the first embodiment can provide a 

video coding technique which attains high noise 
resilience in transmission and can give error 



resilience equivalent to that of conventional 
rectangular picture coding to even arbitrary shape 
picture coding . 

The arrangement and processing on the coding side 
has been explained in detail. Next, the arrangement 
and processing on the decoding side will be explained 
in detail. 

The decoder section will be described. In the 
decoder section according to the first embodiment shown 
in FIG. 5, the output of a demultiplexer section 302 is 
connected to a decoder section 303 and sync detector 
section 304. The output of the sync detector section 
304 is connected to the other input of the decoder 
section 303. The output of the decoder section 303 is 
connected to an error check section 305. The output of 
the error check section 305 is connected to the output 
of the decoder section 303 and an important information 
constructing section 306. The output of the important 
information constructing section 306 is connected to 
the decoder section 303. 

The demultiplexer section 302 demultiplexes a bit 
stream 331 received from the transmission line/storage 
medium 106 into a picture bit stream 332 and another 
data. The sync detector section 304 detects the sync 
signal RM from the bit stream 332 output from the 
demultiplexer section 302. The decoder section 303 
decodes the picture bit stream 332 output after being 
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demultiplexed by the demultiplexer section 302, and 
generates picture data. At this time, the decoder 
section 303 executes decoding processing in synchronism 
with a sync signal detected by the sync detector 
5 section 304 . 

The important information constructing section 306 
obtains data being decoded by the decoder section 303. 
If VOP (Video Object Plane) being decoded by the 
decoder section 303 includes a VOP header, the 

10 important information constructing section 306 extracts 

information of the VOP header, and outputs it to the 
decoder section 303. 

The error check section 305 checks decoded 
information 334 output from the decoder section 303 to 

15 detect whether an error occurs during decoding 

operation. If an error is detected, the error check 
section 305 informs the important information 
constructing section 306 of generation of the error in 
decoding processing so as to stop outputting important 

20 information to the decoder section 303. 

If an error occurs, the decoder section 303 
performs processing corresponding to the error. After 
processing corresponding to the error, the decoder 
section 303 performs decoding operation from the 

25 position of a next sync signal detected by the sync 

detector section 304 . 

In this arrangement, the bit stream 331 received 
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from the transmission line/storage medium 106 is 
demultiplexed into the picture bit stream 332 and 
another data by the demultiplexer section 302. This 
another data is transmitted to a corresponding decoder 
5 section. 

The picture bit stream 332 demultiplexed by the 
demultiplexer section 302 is input to the decoder 
section 303 where the bit stream 332 is decoded. 
During decoding processing, a sync signal is detected 

10 from the bit stream 332 by the sync signal detector 

section 304. 

The error check section 305 checks whether an 
error occurs during decoding operation, from the 
decoded information 334 obtained by decoding processing 

15 of the decoder section 303. If an error is detected, 

processing corresponding to the error is executed by 
the decoder section 303, and decoding operation is done 
at the position of a next sync signal detected by the 
sync detector section 304. 

20 The decoder section 303 determines the type of 

next sync signal. If the signal is the sync signal RM, 
and an error signal 335 is true, the decoder section 
303 obtains VOP header information 343 from the 
important information constructing section 306. 

25 When a VOP header exists in VOP (Video Object 

Plane) being decoded by the decoder section 303, the 
important information constructing section 306 outputs 
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information of the VOP header; and when no VOP header 
exists in VOP being decoded, the important information 
constructing section 306 outputs important information 
so long as the important information is inserted by HEC 
5 within the VP header. 

Decoding processing in the decoder section 303 
uses important information obtained by the important 
information constructing section 306. If a VOP header 
exists in VOP (Video Object Plane) being decoded by the 

10 decoder section 303 in the important information 

obtained by the important information constructing 
section 306, the information of the VOP header is 
output; and if no VOP header exists in VOP being 
decoded, important information is output as far as the 

15 important information is inserted by HEC within the VP 

header. On the coding side, important information 
includes not only important information in conventional 
picture coding but also important information in 
arbitrary shape picture coding. Even if some headers 

20 are destroyed, a picture can be decoded from data 

having normal headers regardless of whether the data is 
obtained by coding a conventional picture or arbitrary 
shape picture- Further, the sync signal prevents 
out-of -synchronization of the video packet VP. That is, 

25 even if the video packet VP starting from the sync 

signal RM has an error before the sync signal RM to 
cause out-of -synchronization, the sync signal RM can 
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establish resynchronization . 

Accordingly, the first embodiment can provide the 
decoding technique of a video coding technique which 
attains high noise resilience in transmission and can 
5 give error resilience equivalent to that of 

conventional rectangular picture coding to even 
arbitrary shape picture coding. 

According to this technique, the noise resilience 
in transmission is attained by transmitting, as header 

10 information, not only important information in 

conventional picture coding but also important 
information for arbitrary shape picture coding in 
arbitrary shape picture coding. On the receiving side, 
it is important how to extract the important 

15 information, transmit it to the decoder section 303, 

and use the important information for decoding 
processing . 

The feature of the first embodiment is, therefore, 
the important information constructing section 306. 

20 The important information constructing section 306 will 

be described in detail with reference to FIG. 6. 

As shown in FIG. 6, the important information 
constructing section 306 comprises a conventional 
picture relating important information constructing 

25 section 307, arbitrary shape coding determination 

section 308, switch sections 309 and 311, and arbitrary 
shape picture relating important information 
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constructing section 310. 

When a VP header is detected in the decoder 
section 303, the conventional picture relating 
important information constructing section 307 decodes 
coding mode information, time reference, and the like 
in information of the VP header, and outputs the 
decoded information . 

The arbitrary shape coding determination section 

308 determines whether a picture being decoded by the 
decoder section 303 is an arbitrary shape picture or 
conventional rectangular picture. The switch sections 

309 and 311 are switched in accordance with the 
determination result. The switch sections 309 and 311 
are system switches for two sections. 

The arbitrary shape picture relating important 
information constructing section 310 decodes important 
information (e.g., picture size and picture position) 
relating to an arbitrary shape picture. For an 
arbitrary shape picture, the switch sections 309 and 
311 are switched to be connected to the arbitrary shape 
picture relating important information constructing 
section 310. Then, important information relating to 
the arbitrary shape picture is reconstructed, and 
supplied to the decoder section 303 in addition to 
important information relating to a conventional 
picture from the conventional picture relating 
important information constructing section 307. Thus, 
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the decoder section 303 can also decode the arbitrary 
shape picture. 

In the important information constructing section 
306 having this arrangement, when the decoder section 
5 303 detects a VP header in an input bit stream, the 

conventional picture relating important information 
constructing section 307 decodes coding mode 
information, time reference, and the like. 

The arbitrary shape coding determination section 
10 308 determines whether a picture being decoded by the 

decoder section 303 is an arbitrary shape picture or 
conventional rectangular picture, and generates a 
control signal corresponding to the determination 
result . 

15 The control signal from the arbitrary shape coding 

determination section 308 controls the switch sections 
309 and 311. For an arbitrary shape picture, the 
arbitrary shape picture relating important information 
constructing section 310 decodes important information 

20 (e.g., picture size and picture position) relating to 

the arbitrary shape picture, prepares the final 
important information 343, and supplies it to the 
decoder section 303 as an output from the important 
information constructing section 306. As far as an 

25 expansion header is set in a header, and includes 

important information relating to an arbitrary shape 
picture, the decoding side can extract the important 



information to supply the important information 
necessary for decoding the arbitrary shape picture to 
the decoder section 303. 

In this manner, the first embodiment can give 
error resilience equivalent to that of conventional 
rectangular picture coding to even arbitrary shape 
picture coding. 

In the first embodiment and the second embodiment 
(to be described later) , "picture size" and "position 
information" must be described in arbitrary shape 
picture coding. Each of these pieces of information is 
expressed by 13 bits in MPEG-4, each of "picture size" 
and "position information" requires horizontal 
information and vertical information, and thus 4 X 
13 bits = 52 bits are required. These bits may be 
large redundant data in transmission at a low bit rate. 
For this reason, the data is transmitted after being 
compressed as much as possible. This method will be 
described . 

The size of the video object plane VOP or the like 
is expressed by 13 bits in MEPG-4 . In many cases, 
however, all the 13 bits are not used. From this, a 
method of expressing the size by a variable length and 
decreasing the number of bits will be considered. 

Basically, the size is expressed by a pair of 
"coded word length" + "value". As shown in FIG. 7, a 
header portion representing the code length and a 
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header in this case. In FIG. 12, each figure 
represents the number of bits, and one horizontal line 
represents 32 bits. In MEPG-4, VW, VH, VHMSR, and 
WMSR are expressed by 13 bits each, and CCED and VSCT 
5 are expressed by 1 bit each. 

Herein, a reserve "Reserve" bit (RV) is inserted 
at last in order to align information into 32 bits. If 
VW, VH, and the like may successively appear like a bit 
stream such as a sync signal, for example, markers (M) 

10 may be inserted between respective values to prevent 

forming a bit stream like a sync signal which must not 
appear, as shown in FIG. 13. The position of the 
marker M is not limited to the boundary between pieces 
of information, and may be inserted at any position so 

15 long as the same rule is established between the 

transmitting and receiving sides. 

A flag representing the presence of an expansion 
header must finally be inserted in conventional header 
information. Thus, 1-bit information representing 

20 whether an expansion header exists in a conventional 

header is inserted. These formats are merely examples. 
Alternatively, header information can be formed from 
only some of these data or a combination with another 
information . 

25 According to the second embodiment, in coding and 

packetting a video picture, an expansion header can be 
added to a packet header for inserting conventional 



picture relating important information. When an 
arbitrary shape picture is to be coded and transmitted, 
its arbitrary shape picture relating important 
information is inserted in the expansion header, added 
as a packet header to data, and packetted. Hence, an 
arbitrary shape picture can be reconstructed in units 
of packets. Even arbitrary shape picture coding can 
attain error resilience equivalent to that of 
conventional rectangular picture coding. Even if some 
VOP headers or VPs are destroyed, a video picture can 
be decoded. 

An example of a decoder section for decoding this 
packet will be described. 

An arrangement of the decoder section will be 
explained. In the decoder section shown in FIG. 14, 
the output of a demultiplexing section 702 for 
receiving a coded bit stream is connected to a decoder 
section 703 and important information construction 
section 705. The output of the decoder section 703 is 
connected to the important information construction 
section 705 via an error check section 704. The output 
of the important information construction section 705 
is connected to the decoder section 703. The 
demultiplexing section 702 performs demultiplexing for 
a bit stream 731 input from the transmission 
line/storage medium 106 into a picture bit stream 732, 
packet header 735, and another data. 
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CLAIMS 

1. A video coding apparatus comprising: 

a coder section for generating coded information 
obtained by coding an input video picture as a bit 
stream; 

an important information reconstructing section 
for extracting, from the coded information obtained by- 
said coder section, important information representing 
a rule of coding a set of predetermined bit streams, 
and reconstructing the important information; 

a sync signal generator section for generating a 
sync signal; and 

a bit stream reconstructing section for adding the 
sync signal output from said sync signal generator 
means and the important information reconstructed by 
said important information reconstructing section to 
the bit stream coded by said coder section, to 
reconstruct the bit stream. 

2 . The video coding apparatus according to 
claim 1, wherein said important information 
constructing section comprises: 

a first important information reconstructing 
section for reconstructing, from the coded information, 
conventional picture relating information as important 
information in a conventional coding scheme of coding a 
frame in units of rectangular regions; 

a second important information reconstructing 
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section for reconstructing, from the coded information, 
arbitrary shape picture relating important information 
as important information in an arbitrary shape coding 
scheme of coding a picture in the frame in units of 
5 arbitrary shape picture regions; 

an arbitrary shape coding determination section 
for determining based on the coded information whether 
the coded picture is an arbitrary shape picture; 

a switch section for outputting the arbitrary 
10 shape picture relating important information when said 

arbitrary shape coding determination section determines 
that the coded picture is an arbitrary shape picture; 
and 

a multiplexer section for multiplexing the 
15 conventional picture relating information and an output 

from said switch section. 

3. The video coding apparatus according to 
claim 2, wherein said arbitrary shape coding 
determination section determines whether the coded 

20 picture is a rectangular picture or an arbitrary shape 

picture to output a determination signal and controls 
said switch section in accordance with the 
determination signal . 

4. The video coding apparatus according to 

25 claim 1, wherein said bit stream reconstructing section 

doubles the important information in a predetermined 
format and insert the doubled important information 
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into a header. 

5. The video coding apparatus according to 
claim 1, wherein said bit stream reconstructing section 
comprises a macroblock boundary determination section 

5 for determining whether or not the coded bit stream is 

positioned on a macroblock boundary, a counter for 
counting bits of the coded bit stream, a sync signal 
inserting determination section for outputting an 
insertion permission signal when the macroblock 

10 boundary determination section determines the 

macroblock boundary for the bit stream and a count 
value of the bits of the bit stream exceeds a given 
value, a header information inserting section for 
forming expansion header information from the important 

15 information and the sync signal and adding the 

expansion information to the coded bit stream in 
response to the insertion permission signal from said 
sync signal inserting determination section. 

6. The video coding apparatus according to 

20 claim 5, wherein the expansion header information is 

provided in a video packet starting at the sync signal. 

7. A video decoding apparatus comprising: 

a bit stream receiving section for receiving a bit 
stream including coded video information that is 
25 obtained by coding a video picture, sync information 

and header information including important information 
representing a rule of coding a set of predetermined 



bit streams, 

a demultiplexer section for demultiplexing the 
coded bit stream into a picture bit stream 
corresponding to the video information; 

a decoder section for decoding the picture bit 
stream; 

a sync signal detector section for detecting a 
sync signal from the bit stream, and informing said 
decoder section of the sync signal; 

an error check section for checking presence of an 
error on the basis of decoded information of said 
decoder section; and 

an important information reconstructing section 
for reconstructing important information from the 
header information output from said decoder section, 
and informing said decoder section of the important 
information when said error check section determines 
absence of an error . 

8 . The video decoding apparatus according to 
claim 7, wherein said important information 
reconstructing section comprises: 

a first important information reconstructing 
section for reconstructing conventional picture 
relating important information from the header 
information ; 

an arbitrary shape coding determination means for 
determining based on the header information whether a 
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picture decoded from the header information is an 
arbitrary shape picture; 

a second important information reconstructing 
means for reconstructing arbitrary shape picture 
5 relating important information from the header 

information; 

a first switch section for inputting the header 
information to said second important information 
reconstructing section when said arbitrary shape coding 

10 determination section determines that the decoded 

picture is an arbitrary shape picture; and 

a second switch section for outputting the 
arbitrary shape picture relating important information 
when said arbitrary shape coding determination section 

15 determines that the decoded picture is an arbitrary 

shape picture. 

9. The video decoding apparatus according to 
claim 7, wherein said important information 
reconstructing section outputs VOP header information 

20 when a VOP header is included in the bit stream 

obtained during decoding of said decoder section, and 
outputs the important information of the VOP when no 
VOP header is included in the bit stream obtained 
during decoding of said decoder section. 

25 10. The video decoding apparatus according to 

claim 9, wherein said important information 
reconstructing section decodes the conventional picture 
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relating important information when the VOP header is 
found from the bit stream in said decoder section. 

11. The video decoding apparatus according to 
claim 7, wherein said decoder section executes a 

5 decoding operation from a position of a next sync 

signal detected by said sync signal detector section 
after performing a processing corresponding to an error 
when said error check section detects the error. 

12. A video coding apparatus comprising: 

10 a coder section for generating coded information 

obtained by coding an input video picture as a bit 
stream; 

an important information reconstructing section 
for extracting, from the coded information obtained by 

15 said coder section, important information representing 

a rule of coding a set of predetermined bit streams, 
and reconstructing the important information; 

a bit stream divider section for dividing the bit 
stream coded by said coder section into a plurality of 

20 divided bit strings; 

a packet header generator section for generating a 
packet header from the important information 
reconstructed by said important information 
reconstructing section; and 

25 a packet constructing section for constructing a 

packet using the divided bit strings and the packet 
header. 
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13. The video coding apparatus according to 
claim 12, wherein said important information 
reconstructing section comprises: 

an arbitrary shape picture relating important 
information reconstructing section for reconstructing 
arbitrary shape picture relating important information 
from the coded information; 

an expansion header insertion determining section 
for determining based on the coded information whether 
an expansion header holding the arbitrary shape picture 
relating important information is to be inserted in the 
packet header; and 

a switch section for inputting the header 
information to the arbitrary shape picture relating 
important information when said expansion header 
insertion determining section determines that the 
expansion header is to be inserted. 

14. The video coding apparatus according to 
claim 12, wherein said packet header generator section 
outputs the packet header containing the important 
information therein to said packet constructing section, 
and said bit stream divider section divides the bit 
stream output from said coder section in a packet size 
and outputs it. 

15. A video decoding apparatus comprising: 

a bit stream receiving section for receiving a bit 
stream including coded video information obtained by 



coding a video picture, sync information, and packet 
header information including important information 
representing a rule of coding a set of predetermined 
bit streams when the video picture is coded; 

a demultiplexer section for demultiplexing the bit 
stream into a picture bit string of the video picture 
information and packet header information; 

a decoder section for decoding the picture bit 
string; 

an error check section for checking presence of an 
error on the basis of decoded information of said 
decoder section; and 

an important information reconstructing section 
for extracting and reconstructing the important 
information from the packet header information, and 
informing said decoder section of the reconstructed 
important information, when said error check section 
determines presence of an error and the important 
information is required in decoding. 

16. The video coding apparatus according to 
claim 15, wherein said important information 
reconstructing section comprises: 

a first important information reconstructing 
section for reconstructing arbitrary shape picture 
relating important information from the packet header 
information; 

an expansion header insertion determining section 
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for determining based on the packet header information 
whether an expansion header is inserted in the packet 
header; and 

a switch section for inserting the packet header 
5 information to the arbitrary shape picture relating 

important information when said expansion header 
insertion determining section determines that the 
expansion header is inserted. 

17. A video transmission system comprising: 
10 a server computer including a video coding 

apparatus for coding a video to generate a coded bit 
stream; 

a transmitter for transmitting the coded bit 
stream of said server computer; 
15 a receiver for receiving the coded bit stream from 

said server computer; and 

a client computer including a video decoding 
apparatus for decoding the coded bit stream from said 
receiver, and wherein 
20 said video coding apparatus comprises: 

a coder section for generating coded information 
obtained by coding an input video picture as a bit 
stream; 

an important information reconstructing section 
25 for extracting, from the coded information obtained by 

said coder section, important information representing 
a rule of coding a set of predetermined bit streams, 




and reconstructing the important information; 

a sync signal generator section for generating a 
sync signal; and 

a bit stream reconstructing section for adding the 
5 sync signal output from said sync signal generator 

means and the important information reconstructed by 
said important information reconstructing section to 
the bit stream coded by said coder section, to 
reconstruct the bit stream, and 
10 said video decoding apparatus comprises: 

a demultiplexer section for demultiplexing the 
received coded bit stream into a picture bit stream 
corresponding to the video information; 

a decoder section for decoding the picture bit 
15 stream; 

a sync signal detector section for detecting a 
sync signal from the bit stream, and informing said 
decoder section of the sync signal; 

an error check section for checking presence of an 
20 error on the basis of decoded information of said 

decoder section; and 

an important information reconstructing section 
for reconstructing important information from the 
header information output from said decoder section, 
25 and informing said decoder section of the important 

information when said error check section determines 
absence of an error. 



78 



18. A video transmission system comprising: 
a server computer including a video coding 
apparatus for coding a video to generate a coded bit 
stream; 

a transmitter for transmitting the coded bit 
stream of said server computer; 

a receiver for receiving the coded bit stream from 
said server computer; and 

a client computer including a video decoding 
apparatus for decoding the coded bit stream from said 
receiver, and wherein 

said video coding apparatus comprises: 

a coder section for generating coded information 
obtained by coding an input video picture as a bit 
stream; 

an important information reconstructing section 
for extracting, from the coded information obtained by 
said coder section, important information representing 
a rule of coding a set of predetermined bit streams, 
and reconstructing the important information; 

a bit stream divider section for dividing the bit 
stream coded by said coder section into a plurality of 
divided bit strings; 

a packet header generator section for generating a 
packet header from the important information 
reconstructed by said important information 
constructing section; and 
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a packet constructing section for constructing a 
packet using the divided bit strings and the packet 
header, and 

said video decoding apparatus comprises: 
5 a demultiplexer section for demultiplexing the 

received bit stream into a picture bit string of the 
video picture information and packet header 
information; 

a decoder section for decoding the picture bit 
10 string; 

an error check section for checking presence of an 
error on the basis of decoded information of said 
decoder section; and 

an important information reconstructing section 
15 for extracting and reconstructing the important 

information from the packet header information, and 
informing said decoder section of the reconstructed 
important information, when said error check section 
determines presence of an error and the important 
20 information is required in decoding. 

19. A recording medium storing: 
a shape information header storing arbitrary 
information of a picture to be recorded; and 

a plurality of VOPs (Video Object Planes) each 
25 including a plurality of macroblocks, each of the 

macroblocks being constructed by a VP header and MB 
data following the VP header, and the VP header 




including important information representing a rule of 
coding a set of predetermined bit streams. 

20. The recording medium according to claim 19, 
wherein the important information of the VP header 

5 includes information about a width of a picture, 

information about a height of the picture, information 
about x-and y coordinates at which the picture is 
pasted, a flag representing whether coding is done 
after the shape information is scaled-down, and 
10 information about a coding type of the shape 

information . 

21. A video decoding apparatus comprising: 
demultiplexer means for demultiplexing an input 

bit stream into a picture stream; 
15 decoder means for decoding the picture bit stream; 

sync signal detector means for detecting a sync 

signal from the picture bit stream, and informing said 

decoder means of the sync signal; and 

important information reconstructing means for 
20 reconstructing important information from header 

information output from said decoder means, and . 

informing said decoder means of the important 

information . 

22. A video coding apparatus according to claim 22, 
25 further comprising: 

means for reconstructing conventional picture 
relating important information from the header 
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information; 

arbitrary shape coding determination means for 
determining based on the header information whether a 
decoded picture is an arbitrary shape picture; 

arbitrary shape picture relating important 
information constructing means for reconstructing 
arbitrary shape picture relating important information 
from the header information; 

switch means for inputting the header information 
to said arbitrary shape picture relating information 
reconstructing means when said arbitrary shape coding 
determination means determines that the decoded picture 
is an arbitrary shape picture; and 

switch means for outputting the arbitrary shape 
picture relating important information when said 
arbitrary shape coding determination means determines 
that the decoded picture is an arbitrary shape picture. 
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